A Bayesian Analysis of Algorithms for Learning Finite Functions
نویسنده
چکیده
We consider algorithms for learning functions f : X ! Y , where X and Y are nite, and there is assumed to be no noise in the data. Learning algorithms, Alg, are connected with ?(Alg), the set of prior probability distributions for which they are optimal. A method for constructing ?(Alg) from Alg is given and the relationship between the various ?(Alg) is discussed. Improper algorithms are identi-ed as those for which ?(Alg) has zero volume. Improper algorithms are investigated using linear algebra and two examples of improper algorithms are given. This framework is then applied to the question of choosing between competing algorithms. \Leave-one-out" cross-validation is hence characterised as a crude method of ML-II prior selection. We conclude by examining how the mathematical results bear on practical problems and by discussing related work, as well as suggesting future work.
منابع مشابه
Comparative Analysis of Machine Learning Algorithms with Optimization Purposes
The field of optimization and machine learning are increasingly interplayed and optimization in different problems leads to the use of machine learning approaches. Machine learning algorithms work in reasonable computational time for specific classes of problems and have important role in extracting knowledge from large amount of data. In this paper, a methodology has been employed to opt...
متن کاملSample size determination for logistic regression
The problem of sample size estimation is important in medical applications, especially in cases of expensive measurements of immune biomarkers. This paper describes the problem of logistic regression analysis with the sample size determination algorithms, namely the methods of univariate statistics, logistics regression, cross-validation and Bayesian inference. The authors, treating the regr...
متن کاملAn Introduction to Inference and Learning in Bayesian Networks
Bayesian networks (BNs) are modern tools for modeling phenomena in dynamic and static systems and are used in different subjects such as disease diagnosis, weather forecasting, decision making and clustering. A BN is a graphical-probabilistic model which represents causal relations among random variables and consists of a directed acyclic graph and a set of conditional probabilities. Structure...
متن کاملImprove Estimation and Operation of Optimal Power Flow(OPF) Using Bayesian Neural Network
The future of development and design is impossible without study of Power Flow(PF), exigency the system outcomes load growth, necessity add generators, transformers and power lines in power system. The urgency for Optimal Power Flow (OPF) studies, in addition to the items listed for the PF and in order to achieve the objective functions. In this paper has been used cost of generator fuel, acti...
متن کاملImpact of Patients’ Gender on Parkinson’s disease using Classification Algorithms
In this paper the accuracy of two machine learning algorithms including SVM and Bayesian Network are investigated as two important algorithms in diagnosis of Parkinson’s disease. We use Parkinson's disease data in the University of California, Irvine (UCI). In order to optimize the SVM algorithm, different kernel functions and C parameters have been used and our results show that SVM with C par...
متن کامل Structure Learning in Bayesian Networks Using Asexual Reproduction Optimization
A new structure learning approach for Bayesian networks (BNs) based on asexual reproduction optimization (ARO) is proposed in this letter. ARO can be essentially considered as an evolutionary based algorithm that mathematically models the budding mechanism of asexual reproduction. In ARO, a parent produces a bud through a reproduction operator; thereafter the parent and its bud compete to survi...
متن کامل